Virtual tissue microstructure reconstruction across species using generative deep learning

Analyzing tissue microstructure is essential for understanding complex biological systems in different species. Tissue functions largely depend on their intrinsic tissue architecture. Therefore, studying the three-dimensional (3D) microstructure of tissues, such as the liver, is particularly fascinating due to its conserved essential roles in metabolic processes and detoxification. Here, we present TiMiGNet, a novel deep learning approach for virtual 3D tissue microstructure reconstruction using Generative Adversarial Networks and fluorescence microscopy. TiMiGNet overcomes challenges such as poor antibody penetration and time-intensive procedures by generating accurate, high-resolution predictions of tissue components across large volumes without the need of paired images as input. We applied TiMiGNet to analyze tissue microstructure in mouse and human liver tissue. TiMiGNet shows high performance in predicting structures like bile canaliculi, sinusoids, and Kupffer cell shapes from actin meshwork images. Remarkably, using TiMiGNet we were able to computationally reconstruct tissue structures that cannot be directly imaged due experimental limitations in deep dense tissues, a significant advancement in deep tissue imaging. Our open-source virtual prediction tool facilitates accessible and efficient multi-species tissue microstructure analysis, accommodating researchers with varying expertise levels. Overall, our method represents a powerful approach for studying tissue microstructure, with far-reaching applications in diverse biological contexts and species.


Dear Emily Chenette,
We sincerely appreciate the thoughtful and detailed feedback on our manuscript from you and the reviwers.We have carefully considered each of the comments and have made the necessary revisions to address them.Below is a point-by-point response to each of the reviewers' comments: Reviewer #1: 1. What is the training and testing framework of TiMiPNet, TiMiGNet, and TiMiGNet+?Figure 1 only shows a training framework of TiMiGNet.What is the input and output of these networks?
Thank you for pointing this out.We have added a new figure (S1_Fig A) illustrating the training framework for TiMiPNet.Additionally, we have provided explanations on the differences in the training frameworks of TiMiGNet and TiMiGNet+ (lines 188 to 195).The input for the networks are 64x64 or 128x128 image patches of membranes, and the output for the networks are 64x64 or 128x128 image patches of bile canaliculi.Corresponding figures (S1_Fig B, C) showing the testing/prediction frameworks have also been included.
2. Lack of comparative experiments for predictor and generator.Please compare with other deep learning methods.
We agree that a comparison of several deep learning methods is needed and have included them.Specifically, we used Unet (TiMiPNet), CycleGAN (TiMiGNet), and added Pix2Pix to our analysis.Figures F4 Sup a,b demonstrate similar performance across architectures for predicting BC structures.We have expanded our specific case with CycleGAN by modifying the loss function (TiMiGNet+), achieving better results for large structures like sinusoids (F3 Sup).We also evaluated the generator performance for CycleGAN-like architectures as suggested (F4 Sup c,d).

Please use a table to show the average PSNR and average SSIM on the complete testing dataset.
We have added a table displaying the average PSNR and SSIM for the complete testing dataset.
4. The pixels of the data picture are too low, and the text in some pictures can't be seen clearly.
We have exported all images in higher quality and increased the text size for clarity.

Please indicate the number of images used for training and testing.
We apologize for the lack of clarity.The number of images and patches used for training and testing are now described in the "Model Training" and "Quality Evaluation Metrics" sections of the Methods.
6. Please indicate the PSNR and SSIM under each sample in each figure .We have added the average PSNR and SSIM values for each predicted image.
7. In short, the logic of the paper needs to be reorganized.There are some parameters that need to be explained in detail.
Thank you for the comment.We have reorganized the text for better logical flow and provided detailed explanations for relevant parameters.

Reviewer #2:
1.A quantitative evaluation was conducted, but quantitative numbers were not mentioned in the results section.Specific descriptions of these are needed.
Thank you for the comment.We clarified this by adding a summary table (S1 Table ) of the average algorithm performance results and incorporated specific quantitative numbers into the results section.

Is it possible to compare the performance of the proposed model with the default CycleGAN?
We performed a detailed comparison as explained in our response to Reviewer #1, point 2.
3. What diseases can be better discovered using the proposed algorithm?The clinical significance needs to be further elucidated.
Thank you for raising this point.We have added a paragraph discussing the clinical significance and the potential diseases that can be better discovered using the proposed algorithm (lines 496 to 508).
4. There are disadvantages of GAN.The following paper introduces the shortcomings."Application of generative adversarial networks (GAN) for ophthalmology image domains: a survey" Were there any experimental results like this? Discuss further the disadvantages.
Thank you for raising this point.We have addressed the disadvantages of GANs, as discussed in "Application of Generative Adversarial Networks (GAN) for Ophthalmology Image Domains: A Survey".We included a discussion on mode collapse, spatial deformity, unintended changes due to domain differences, and checkerboard artifacts in the final section of the manuscript (lines 510 to 521).

Reviewer #3:
The manuscript is well written and structured in a clear and logical way to present the new TiMiGNet solution.The authors discussed the methods, results, and debates thoroughly and thoughtfully.The conclusions are strongly supported by the comprehensive evidence.I like the idea of using TiMiGNet for biotissue image recognition and segmentation.The novel approach overcomes some of the existing challenges of biotissue image segmentation.I recommend the editor to accept this manuscript for publication.
One minor edit for authors is to replace all graphs with high-resolution images.
We appreciate your positive feedback and recommendation for publication.We have replaced all graphs with high-resolution images as suggested.
We hope these revisions adequately address all your comments and improve the quality and clarity of our manuscript.Thank you for your valuable feedback and consideration.